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DETAILED ACTION 
Continued Prosecution Application 

1 . The request filed on 05/27/2004 for a Requested for Continued Examination 
(RCE) under 37 CFR 1.53(d) based on parent Application No. 09/670635 is acceptable 
and a RCE has been established. An action on the RCE follows. 

2. Claims 1-27 and 58-65 are pending in this application. 

Claim Rejections - 35 USC § 101 
35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

3. Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. The method in claim 1 which does not required 
to be performed by the computer system, the method can be done on the papers. 

Claim 1-13 are depended on claim 1; therefore, they are rejected under the same 
reason. 

Claim 58 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. The method in claim 1 which does not required 
to be performed by the computer system, the method can be done on the papers. 

Claims 59-61 are depended claims; therefore, they are rejected under the same 
reason. 
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Claim 62 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. The method in claim 1 which does not required 
to be performed by the computer system, the method can be done on the papers. 

Claim 63-65 are depended claims; therefore, they are rejected under the same 
reason. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-27 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

Shneiderman et al. (Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos, 

July 2000) in view Mizoguchi Yoshiyuki (EP 0 678 816 A2). 

Regarding on claims 1,16 and 17, Shneiderman teaches a method or archiving 
and retrieving digital media items, comprising: 

Receiving a user input identifying a group of users to which an archiving belongs 
(the user log on to the system) (fig. 4); 

Receiving archiving input data identifying: a digital media item to be archived for 
the group the group (add photo) (fig. 4), the user's selection of zero or more group event 
types from a predetermined plurality of group event types specific to the group 
(visualizing personal histories workshop July 1997) (fig. 4), the user's selection of zero 
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or more persons in the group (loskowski, Sharon, Li, Jia and Plaisant, Catherine) (fig. 
4), and the user's selection of period (the starting date and the ending date) (fig. 4); 

Generating index information using the received user archiving input (each photo 
should have a unique reference and photos with the same reference are not allowed to 
the stored in the in this table even through they have different attribute values) (col. 6, 
lines 14-17); 

Repeating the reception of archiving input data, the generation of the index 
information and the storing of the index information for a plurality of digital media items 
(the processing continue for all other picture in my document) (fig. 4) 

Shneiderman does not explicitly teach receiving retrieval input data representing 
a selection of a default or zero or more group event types from the predetermined 
plurality of group event types for the group, a selection of a default or zero or more 
persons in the group, a selection of time period; and using the selections and the 
identified group to retrieve and output digital media items that match the selection. 
However, Mizoguchi discloses "in the display state show in Fig. 9B, when one of the 
"person key", "place" key, and "other data" key in the upper right portion 7b is operated, 
a list of person information, place information, and other information stored as the 
associated data of the image data in the memory card MC are displayed in accordance 
with the operated search key. When given information "X" is operated in the list, images 
having the information "X" as the associated information are searched and displayed 
from the last one" (col. 10, lines 19-28). This suggest the user selecting the list of user, 
place and other data for a query. Therefore, it would have been obvious to one ordinary 
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skill in the art at the time of the invention was made to modify Shneiderman's system to 
include user selecting the list of user, place and other data as taught by Mizoguchi in 
order to retrieve media with different parameters. 

Regarding on claims 2 and 18, Shneiderman teaches a method recited in claim 1 
wherein the retrieval input data comprises a user input from user identifying a group to 
which the user belongs and the digital media items are retrieved using the group 
identified for the other user in the user retrieval input (user log in with the system which 
identify the person in the library) (fig. 4). 

Regarding on claims 3 and 19, Shneiderman teaches a method recited in claim 1 
including defining the distinct groups of people, and defining group event types that are 
appropriate for members of the groups to distinguish episodic events memorable to the 
group (each of the photo being association with group of users in the photo) (fig. 4). 

Regarding on claims 4 and 20, Shneiderman teaches a method recited in claim 1 
including receiving said digital media item to be archived, and storing said digital media 
item in association with the index information (col. 2, lines 14-17). 

Regarding on claims 5 and 21, Shneiderman teaches the method recited in claim 
1 including receiving archiving input data identifying a digital media item as being 
associated with a memorable high point in the mind of the user (time is the high point of 
the photo) (fig. 4). 

Regarding on claims 6 and 22, Shneiderman teaches the method recited in claim 
5 wherein the retrieval input data includes an input selecting memorable high points 
(searching by names of people in each photo) (col. 3, lines 8-9). 
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Regarding on claims 7 and 23, Shneiderman teaches the method recited in claim 
1 wherein the index information includes an identification of a media type of the digital 
media item (col. 3, lines 4-8). 

Regarding on claims 8 and 24, Shneiderman teaches the method recited in claim 
7 wherein the retrieval input data includes an input identifying a media type, and the 
digital media items are retrieved and output based on the identified media type (col. 1 , 
lines 4-8). 

Regarding on claims 9 and 25-26, Shneiderman teaches the method recited the 
method in claim 1 including receiving archiving input data identifying a plurality of digital 
media items and an input identifying the digital media items to be associated as 
perceived by the user, wherein the index information is generated to include the 
identified association (the association with people, month and person) (fig. 6 in page 7). 

Regarding on claim 10, Shneiderman teaches the method recited in claim 9 
wherein when digital media items are retrieved and output as a result of the user 
retrieval input, any digital media items having the identified association in the index 
information are automatically identified for retrieval and output (page. 3, col. 2, lines 8- 
10). 

Regarding on claim 11, Shneiderman teaches the method recited in claim 10 
wherein the automatically identified digital media items are automatically retrieved and 
output (page. 3, col. 2, lines 8-10). 

Regarding on claim 12, Shneiderman teaches a method recited in claim 10 
including outputting a notification to a user that associated digital media items are 



Application/Control Number: 09/670,635 Page 7 

Art Unit: 2162 

available, and retrieving and outputting automatically identified digital media items in 
response to a user input page. 3, col. 2, lines 8-10). 

Regarding on claims 13 and 27, Shneiderman teaches the method recited in 
claim 1 further comprising: 

Receiving a user request for automatic nostalgic retrieval (page. 3, col. 2, lines 8- 

10), 

Automatically generating an initial set of said selections (page. 3, col. 2, lines 8- 

10), 

Using the modified selections to retrieve and output digital media items (page. 3, 
col. 2, lines 8-10); and 

Repeating the modifying, and retrieval and output steps (page. 3, col. 2, lines 8- 

10). 

Claims 14-15 are rejected under the same reason as claim 1. 

5. Claims 58-65 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Shneiderman et al. (Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos, 
July 2000) in view Astle (US. Patent No. 5,485,611). 

Regarding on claim 58, Shneiderman teaches a media archival method, under 
control of an operator who is a member of a group: 

Authenticating an operator as a member of a group of users (the user log in to 
the system to identify the photo in the samplehcilibraryl. Mbb) (fig. 4), 
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Identifying candidate identification values based upon the group with whom the 
operator is authenticated (one the user log in the C:\my photosV.. the images and 
groups of people are identified) (fig. 4), 

Querying the operator for selection of identification data to be associated with a 
digital media item, the query identifying the candidate identification values and including 
valid selections of an event type and persons from the group and time (after the log 
process successful, the system displays graphical user interface to allow the user to 
select people, date, location and description) (fig. 4). Shneiderman does not explicitly 
teach generating index information from a response of the operator, and storing the 
index information in association with the digital media item. However, Shneiderman 
discloses "each photo should have a unique reference and photos with the same 
references are not allowed to be stored in this table even through they have different 
values" (col. 6, lines 14-17). On the other hand, Astle discloses "an index can be 
prepared by the consumer while viewing the video footage stored within the database. 
The location on the particular video cassette may be denoted by a time index or a 
counter index, for example. Thus, the user-prepared index may indicate that he hot-air 
balloon event appears on video cassette number 12, starting at time 1:17:23 (in hours: 
minutes: seconds format) from the beginning of the video cassette, and/or at counter 
number 2351 from the beginning of the tape" (col. 2, lines 16-25). This suggests that 
the index is created with event and time. Therefore, it would have been obvious to one 
ordinary skill in the art at the time of the invention was made to modify Shneiderman's 
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system to include the creating of an index utilizing the event and time as taught by Astle 
in order to retrieve the image using different parameters. 

Regarding on claims 59 and 63, Shneiderman teaches the archived method 
recited in claim 58, wherein the candidate identification values for persons include 
names of group members (selecting names in the library) (fig. 4). 

Regarding on claims 60 and 64, Shneiderman teaches the archival method 
recited in claim 58, wherein the stored index information include names of group 
members (history visualization workshop) (fig. 4). 

Regarding on claims 61 and 65, Shneiderman teaches the archival method 
recited in claim 58, wherein the stored index information includes a flag that 
distinguishes high point items from other items, and the method further comprises 
setting the flag if the operator response includes an indication that the digital media item 
is a high point (show name labels) (fig. 4). 

Claim 62 is rejected under the same reason as claim 58. 

Conclusion 

6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Baoquoc N. To whose telephone number is at 571-272- 
4041 or via e-mail BaoquocN.To@uspto.gov. The examiner can normally be reached 
on Monday-Friday: 8:00 AM -4:30 PM, EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Breene can be reached at 571-272-4107. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 305- 
3900. 

Any response to this action should be mailed to: 
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Commissioner of Patents and Trademarks 
Washington, D.C. 20231. 

The fax numbers for the organization where this application or proceeding is 
assigned are as follow: 

(703) 872-9306 [Official Communication] 



Baoquoc N. To 
January 21, 2005 



